In-house discv5#11728
Conversation
…nto feature/kademlia-discv4
# Conflicts: # src/Nethermind/Nethermind.Runner/configs/hoodi_archive.json # src/Nethermind/Nethermind.Runner/configs/mainnet_archive.json # src/Nethermind/Nethermind.Runner/configs/sepolia_archive.json
…SZ ulong limits, injectable discv5 ENR filter) (#11985) * feat(ssz): support list limits beyond int range in the SSZ generator Beacon-chain lists use limits up to 2^40 (VALIDATOR_REGISTRY_LIMIT), which overflowed the int-typed SszListAttribute. The limit is now ulong end to end; decode-side count guards are omitted when the limit exceeds int.MaxValue since an int-typed count can never violate them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(discv5): make the consensus-only ENR drop policy injectable The shared discv5 stack hard-dropped consensus-only node records in the response handler, adapter acceptance check, and node source — fatal for a consumer that wants CL peers. Discv5RecordFilter makes the policy injectable with the execution-layer behavior as the default (unchanged); a beacon-chain scope can register AcceptAll. Needed by the embedded beacon chain driver plugin (#11976). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(discv5): dedicated IDiscv5RecordFilter, required everywhere Replace the bool-parameterized filter class with an interface and two implementations, and make the dependency non-optional at every record-handling component (no null-coalesced defaults) so each discv5 instance states its policy explicitly. The interface documents why the policy exists: discv5 is one DHT shared by EL and CL nodes, and which records are useful is per instance — the EL must drop consensus-only ENRs it can never dial over RLPx, while a CL instance exists precisely to find them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ssz): lift the bitlist limit guard to ulong; drop a dead generator arm Review follow-up: ValidateSszBitlistLimit now takes ulong like the list variant, the BitList validation emission carries the UL suffix, and the bitlist Encode emission clamps to int.MaxValue (the runtime parameter is int-typed and unused). The Kind.List-with-BitArray arm could never fire — such properties resolve to Kind.BitList — and is removed. Regression: the huge-limit test now also covers a 2^40-limit bitlist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(ssz): reject huge bitlist limits; address review feedback - Generator's SszListAttribute mirror now takes `ulong limit`, matching the runtime attribute (flcl42). - SszProperty parses the limit as `ulong` directly (the attribute is `ulong`, so the value is never `int`) and now rejects a `BitArray` list whose limit exceeds `int.MaxValue` with an SSZ003 generator error instead of silently clamping the int-typed Encode parameter — a BitArray cannot hold more than int.MaxValue bits (flcl42). - Drop the now-unsupported huge-limit bitlist test type; add a generator diagnostic regression test for the rejection. - Trim the IDiscv5RecordFilter <remarks> block (flcl42). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
LukaszRozmej
left a comment
There was a problem hiding this comment.
I don't like LruCache changes
# Conflicts: # src/Nethermind/Nethermind.Init/Steps/InitializeNetwork.cs # src/Nethermind/Nethermind.Network.Discovery/DiscoveryApp.cs # src/Nethermind/Nethermind.Network.Discovery/DiscoveryConnectionsPool.cs # src/Nethermind/Nethermind.Network.Discovery/Discv4/Kademlia/KademliaAdapter.cs # src/Nethermind/Nethermind.Network.Discovery/Discv5/DiscoveryV5App.cs
|
@claude reivew all against master |
|
Claude finished @flcl42's task in 8m 39s —— View job PR Review: In-house discv5 — Full Review vs Master
|
# Conflicts: # src/Nethermind/Nethermind.Network.Discovery.Test/E2EDiscoveryTests.cs # src/Nethermind/Nethermind.Network.Discovery/DiscoveryApp.cs # src/Nethermind/Nethermind.Network.Discovery/Discv4/DiscV4KademliaModule.cs

Changes
Types of changes
What types of changes does your code introduce?
Testing
Requires testing
If yes, did you write tests?
Remarks
Change list
Nethermind.Kademlia: routing table, lookup algorithm, node health tracking, distance abstractions, and generic node/key operators now live outside discv4-specific code.FINDNODE/NEIGHBORS, ENR request/response, and XDC extension hooks in the discv4 layer.FINDNODEdistance requests map toGetAllAtDistance, while discv4 keeps nearest-node lookup semantics.eth2records before execution peer admission, skipped malformed records, and prevented stale ENR sequence downgrades.Architecture overview
The new shape separates reusable Kademlia table mechanics from protocol transport.
Nethermind.Kademliaowns generic routing, lookup, health tracking, and distance services;Nethermind.Network.Discoverysupplies Ethereum discovery-specific adapters, persistence, node sources, and lifecycle. Discv4 and discv5 each create their own protocol child scope and own Kademlia instance, then bind protocol-specific message senders and node-source behavior on top of the same core. Discv4 keeps signed RLP packet handling, bonding, and nearest-neighbour discovery, while discv5 owns packet/session encryption, ENR-based messages, and distance-bucketFINDNODEsemantics. Both versions feed discovered execution peer candidates through role-specific ENR conversion so UDP discovery endpoints stay in routing and TCP endpoints go to the peer pool.High-level architecture
flowchart TB NetworkConfig["Network config<br/>DiscoveryVersion + bootnodes"] --> Composite["CompositeDiscoveryApp"] Composite --> V4App["Discv4 DiscoveryApp"] Composite --> V5App["Discv5 DiscoveryV5App"] V4App --> V4Scope["discv4 child scope"] V5App --> V5Scope["discv5 child scope"] V4Scope --> V4Adapter["discv4 KademliaAdapter<br/>bonding + FINDNODE nearest lookup"] V5Scope --> V5Adapter["discv5 KademliaAdapter<br/>session-aware FINDNODE distances"] V4Adapter --> Kad4["Kademlia<PublicKey, Node><br/>discv4 instance"] V5Adapter --> Kad5["Kademlia<PublicKey, Node><br/>discv5 instance"] Kad4 --> Core["Nethermind.Kademlia<br/>routing table + lookup + health"] Kad5 --> Core V4App --> V4Handler["NettyDiscoveryHandler<br/>signed RLP UDP packets"] V5App --> V5Handler["NettyDiscoveryV5Handler"] V5Handler --> PacketCodec["PacketCodec<br/>WHOAREYOU + sessions + AES-GCM"] PacketCodec --> MessageCodec["MessageCodec<br/>PING/PONG/FINDNODE/NODES/TALK"] V4Adapter --> NodeSource4["discv4 NodeSource"] V5Adapter --> NodeSource5["discv5 NodeSource"] NodeSource4 --> PeerPool["PeerPool candidates"] NodeSource5 --> PeerPoolKey moment: discv5 packet handling
sequenceDiagram participant UDP as NettyDiscoveryV5Handler participant Codec as PacketCodec participant Adapter as Discv5 KademliaAdapter participant Kad as Kademlia UDP->>UDP: cheap size checks and bounded queue admission UDP->>Codec: accepted packet bytes Codec->>Codec: decode ordinary, WHOAREYOU, or handshake packet alt no usable session Codec-->>UDP: send WHOAREYOU challenge else authenticated message Codec->>Adapter: Discv5Message Adapter->>Kad: mark inbound health / add or refresh node alt FINDNODE Adapter->>Kad: GetAllAtDistance(requested distances) Kad-->>Adapter: matching nodes Adapter-->>Codec: NODES responses with ENRs else PING/PONG/TALK Adapter-->>Codec: protocol response or handler completion end Codec-->>UDP: encrypted response packet endKey moment: ENR endpoint separation
flowchart LR ENR["ENR"] --> Validate["validate secp256k1<br/>node id, routability, sequence"] Validate --> Eth2{"consensus-only<br/>eth2 record?"} Eth2 -- yes --> Reject["reject for execution discovery"] Eth2 -- no --> Record["NodeRecord"] Record --> Routing["TryFromDiscoveryEnr<br/>UDP endpoint"] Routing --> KadTable["discv5 Kademlia table"] Record --> Candidate{"has usable TCP endpoint?"} Candidate -- yes --> PeerNode["TryFromEnr<br/>TCP endpoint"] PeerNode --> PeerPool["PeerPool"] Candidate -- no --> SkipPeer["skip RLPx peer candidate"] Record --> Sequence{"newer ENR sequence?"} Sequence -- yes --> Cache["replace known ENR"] Sequence -- no --> Keep["keep newer cached ENR"]Key moment: receive-path hardening
flowchart TD Packet["UDP packet"] --> Cheap["read size/type from buffer"] Cheap --> Valid{"valid enough<br/>for this protocol?"} Valid -- no --> DropOrForward["drop or forward without copy"] Valid -- yes --> Limit["global + per-IP limits"] Limit --> Accepted{"within limits<br/>and queue capacity?"} Accepted -- no --> Drop["drop before packet copy/expensive work"] Accepted -- yes --> Queue["bounded channel"] Queue --> Workers["limited dispatch workers"] Workers --> Decode["deserialize / decrypt accepted traffic"] Decode --> Adapter["protocol adapter"] Adapter --> Kad["Kademlia + response handlers"]